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(54) Title: AN IMPROVED SYSTEM FOR CODING SIGNALS 
(57) Abstract 

A method and apparatus for coding an information signal 
are provided. In accordance with one aspect of the invention, 
the method includes the step of encoding all or a portion of 
the information signal with a first encoder to generate a first 
set of redundant bits (preferably ri bits). The method further 
includes the step of passing a portion of (and possibly all) the 
information signal through a structured interleaver to generate 
an interleaved signal. The method then encodes all or a portion 
of the interleaved signal with a second encoder to generate a 
second set of redundant bits (preferably ti bits). Finally, the 
method includes the step of concatenating the information signal, 
the first set of redundant bits, and the second set of redundant 
bits to form an encoded output signal. A significant aspect of 
the present invention is the use of a structured interleaver in 
the encoder. It has been found that the structured interleaver 

provides a low bit error rate, and a much shorter length (and thus delay) than random interleavers. The foregoing concept apply equally 
to multi-level coding, wherein a parallel concatenated code defined by a structured interleaver may be utilized as a constituent code in a 
multi-level encoder. 
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A fundamental axiom which has motivated the development of coding methods, 
known as Shannon's coding theorem, provides that if a source generates information at 
a rate that is less than the capacity of a transmission channel, there exists some 
encoding procedure that will permit the source output to be transmitted through the 
5 channel with an arbitrarily small probability of error. Multilevel coding of signals is 
one coding method known for achieving increased transmission speeds. However, 
multilevel signals are more likely to be corrupted by channel noise. Therefore;, error- 
correcting schemes are generally implemented in multilevel coding systems. 



10 coding to further improve the transmission efficiency, which operates to time-disperse 
bits or symbols before encoding and transmission. Often, several "important" source 
bits are produced in immediate succession. Since a burst error, such as a channel fade, 
could corrupt all of these adjacent bits, it is desirable to tim.e-disperse adjacent 
information bits. Moreover, by rearranging the order of such bits in connection with 

15 the redundant encoding, further noise immunity is achieved. 

As a further improvement, a parallel concatenated coding method, also known 
as turbo coding, has been developed. In its simplest form, a parallel concatenated 
coding scheme includes two simple constituent systematic encoders linked by an 
interleaver. The input bits to the first encoder are scrambled by the interleaver before 

2 0 entering the second encoder. The codeword of the parallel concatenated code 

generally consists of the input bits to the first encoder followed by the parity check bits 
of both encoders, although this construction can be generalized to any number of 
constituent codes. 



As is further known, interleaving is a method used in connection with channel 
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AN IMPROVED SYSTEM FOR CODING SIGNALS 

CROSS-REFERENCE TO RELATED APPLICATIONS 

This application claims the benefit of U.S. Provisional Application Serial No. 
5 60/026,053, filed September 13, 1996, and entitled METHOD FOR ENCODING 
CONCATENATED CODES. 



BACKGROUND OF THE INVENTION 
Field of the Invention 

10 The present invention generally relates to systems for signal coding, and more 

particularly to a signal coding system utilizing a unique parallel concatenated coding 
method for achieving improved bit error rates. 



Discussion of the Related Art 

15 Signal coding, or channel coding, has long been know as a way to protect the 

integrity of data transmitted through a channel. Through the implementation of 
coding, unsystematic redundancy can be removed from message signals so that 
channels can be used with maximum efficiency. In addition, through the use of coding, 
systematic redundancy can be introduced into the transmitted signal so that errors 

2 0 caused by noisy channels can be corrected. Stated another way, the ability to detect 
and/or correct errors resulting from a noisy transmission channel can only be provided 
by the additional transmission of redundant bits, and thus by lowering the effective 
information rate for a given transmission bandwidth. 



1 



wo 98/11671 PCT/US97/16106 

It has been demonstrated that parallel concatenated schemes employing 
convolutional codes as constituent codes, in connection with an iterative decoding 
algorithm, yield coding gains very close to the their theoretical limits (/.e.. Shannon 
limit). However, in the parallel concatenated codes known by the Applicant, the 
5 enhanced performance achieved due to the tremendous reduction in error coefficients, 
has been obtained by utilizing a very long random interleaver. As with any long code 
word, these parallel concatenated codes generally suffer at least one important 
shortcoming; namely, the delay due to the long random interleaver and to the iterative 
decoding. As a result, this type of coding realizes intolerable delays for many 

10 applications. A further shortcoming relates the complexity of maximum-likelihood 

(ML) decoding algorithms, whereby the decoding complexity increases with increasing 
length of the interleaver. 

Various methods and apparatus for coding signals are known in the art, and are 
described in various publications including, "Near Shannon Limit Error Correcting 

15 Coding and Decoding: Turbo-Codes," C. Berrou, A. Glavienux and P. Thitimajshima, 
Proc, of the ICC '93, Geneva, Switzerland, pp. 1064-1070, May 1993, "Unveiling 
Turbo Codes: Some Results on Parallel Concatenated Coding Schemes," S. Benedetto 
and G- Montorsi, IEEE Trans, Information Theory, vol. IT-42, pp. 409-428, March 
1996, "Iterative Decoding of Binary Block and Convolutional Codes," J. Hagenauer, 

20 IEEE Trans. Information Theory, vol IT-42, pp. 429-445, March 1996, 'Turbo Codes 
for PCS Applications," D. Divsalar and F. Pollara, Proc, of the ICC '95, Seattle, 
Washington, June 1995, "Multilevel Codes Based on Partitioning," G. J. Pottie and D. 
P. Taylor, IEEE Trans, Information Theory, vol. IT-35, pp. 87-98, January 1989, 
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"Multilevel Coding Method Using Error-Correcting Codes," H. Imai and S. Krakawa, 
IEEE Tram. Information Theory, vol. IT-23, pp. 371-377, May 1977, and "Channel 
Coding with Multilevel/Phase Signals," G. Ungerboeck, IEEE Trajis. Information 
Theory, vol. IT-28, pp. 55-67, January 1982, which are all hereby incorporated by 
5 reference. 

SUMMARY OF THE INVENTION 

Accordingly, it is a primary aim of the present invention to provide a coding 
system achieving improved efficiencies. Certain objects, advantages and novel features 

10 of the invention will be set forth in part in the description that follows and in part will 
become apparent to those skilled in the art upon examination of the following or may 
be learned with the practice of the invention. The objects and advantages of the 
invention may be realized and obtained by means of the instrumentalities and 
combinations particularly pointed out in the appended claims. 

15 To achieve the advantages and novel features, the present invention is generally 

directed to a method for coding an information signal. In accordance with one aspect 
of the invention, the method includes the step of encoding the information signal with a 
first encoder to generate a first set of redundant bits (preferably ri bits). The method 
further includes the step of passing the information signal through a structured 

2 0 interleaver to generate an interleaved signal. The method then encodes the interleaved 
signal with a second encoder to generate a second set of redundant bits (preferably T2 
bits). Finally, the method includes the step of concatenating the information signal, the 
first set of redundant bits, and the second set of redundant bits to form an encoded 
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output signal. 



The present invention provides a substantial improvement over the parallel 



concatenated codes known in the prior art, which utilize long random interleavers, and 
by providing relatively short, structured interleavers. As a result, an increased 
5 minimum Hamming distance is obtained. For low BERs (which is desired BERs in 
many applications), the performance of the parallel concatenated codes with a 
structured interleaver is better than that obtained with a random interleaver. Another 
important advantage of the structured interleaver is the short required length. A short 
interleaver introduces only a short decoding delay, where in the case of parallel 
10 concatenated code, short interleavers may have additional advantage. In this regard, 
short interleavers may reduced the decoding complexity and the required memory, 
since a soft out soft in decoding algorithm should be implemented for the constituent 
codes. 



15 ability to submit less than all information bits to a given encoder. While it is generally 
desirable to encode all information bits, such encoding need not be performed at a 
single encoder. It will be appreciated that the same does not hold true for the parallel 
concatenated coding apparatus of the prior art. Specifically, when a random 
interleaver is used, at least one encoder must encode all of the information bits. 

2 0 Otherwise, there is a statistical probability that one or more of the information bits will 
not be encoded. In contrast, by using a structured in accordance with the present 
invention, the interleaver can be designed to ensure that all information bits get 



Another significant advantage made possible by the present invention is the 
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encoded, even though only a portion of the information bits may be submitted to each 
encoder. 

In accordance with a further aspect of the invention, parallel concatenated 
codes are used as component codes in multilevel codes. As the first component code 
5 C| a parallel concatenated convolutional code (PCCC), with a reverse block 
interleaver, is used. It is found that a coding gain of more than 5.5 dB may be 
achieved when the parallel concatenated code Ci includes two 8-state, (/.e., M = 3) 
convolutional codes, each of rate one-half The rate of the resulting parallel 
concatenated code is a one-third binary code. Additional improvement in the 

10 performance is obtained by using 16-state constituent codes and a punctured code. In 
this case, the rate of the parallel concatenated code can be increased to one-half (/.e., 
an increase of 1/6 can be obtained in the number of information bits per symbol). 
Namely, an additional 1 dB of coding gain can be obtained by using punctured codes, 
Depdening upon the performance needed, a suboptimal iterative decoder may be 

15 employed. Although a suboptimal iterative decoder will generally degrade the 
performance, it substantially reduces the computational complexity. Base on 
simulation results, it is found that the performance degradation is usually very small, 
especially for small BERs. 

2 0 DESCRIPTION OF THE DRAWINGS 

The accompanying drawings incorporated in and forming a part of the 
specification, illustrate several aspects of the present invention, and together with the 
description serve to explain the principles of the invention. In the drawings: 
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FIG. 1 is a block diagram illustrating a multilevel encoder, as is known in the 
prior art; 

FIG. 2 is a block diagram illustrating a multistage decoder, as is known in the 
prior art; 



parallel concatenated code, as is known in the prior art; 

FIG- 4A is a block diagram of a non-systematic convolutional encoder, as is 
known in the prior art; 

FIG. 4B is a block diagram of a systematic convolutional encoder, as is known 
10 in the prior art; 

FIG. 5 is a block diagram of a two-stage decoder for decoding a parallel 
concatenated coded signal, as is generated by the encoder of FIG. 3; 

FIG. 6 is a block diagram of a two-state decoder similar to that of FIG. 5, but 
further employing a feedback loop for improved performance; 
15 FIG. 7 is a diagram of a matrix illustrating a block interleaver; 

FIG. 8 is a diagram of a matrix illustrating a reverse block interleaver; 

FIGS. 9-12 are diagrams illustrating information sequences for reverse block 
interleavers; and 



FIG. 3 is a block diagram of a simple two level encoder for generating a 



FIGS. 13-17 are computer generated plots illustrating various simulation 



2 0 results comparing bit error rates as a function of the signal to noise ratio. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

Having summarized the present invention, reference v^ill now be made in detail 
to the description of the invention as illustrated in the drawings. While the invention 
will be described in connection v^th these drawings, there is no intent to limit it to the 
5 embodiment or embodiments disclosed therein. On the contrary, the intent is to cover 
all alternatives, modifications and equivalents included wdthin the spirit and scope of 
the invention as defined by the appended claims. 

Backgrou nd on Multilevel Encoding and Multistage Decoding 

Before discussing the preferred embodiment of the present invention, a 

10 backgound on multilevel encoding and multistage decoding will first be provided. In 
this regard, reference is made to FIG. 1, which is a block diagram illustrating the 
structure of a simple multilevel encoder 10. The illustrated encoder is an L level 

^ pu,iu.iiv/i wwtivwic;! <ii ulc xiuiiL cilu. /\s IS Known, tne 

serial to parallel converter 12 includes an input and a plurality of outputs. Most of the 

15 outputs are associated with, and directed to an encoder 14, 16, 17. Each of the 

encoders 14, 16, 17 will receive one or more bits for encoding, and output the encoded 
bits to a symbol selector 18. As illustrated, uncoded bits may be communicated 
directly from the serial to parallel converter 12 to the symbol selector 18 across one or 
more lines 19. 

2 0 Multilevel coding has been recognized as an efficient way of combining error 

correction coding with modulation. Multilevel coding, is generally a combination of 
several error correction codes applied to subsets of some signal constellation. As is 
known, a typical code of this kind is, based on a set So and an L level partition chain 
So/Si.../Sl. In the multilevel encoder 10 of FIG. 1, the sequence of bits associated with 

2 5 the partition is a codeword of the binary code Ciiri,, ky), and Ei is the 
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corresponding encoder. The multilevel code may be made up of convolutional 
component codes, block codes, or a combination thereof. 

An important parameter of a coded modulation scheme is the computational 
complexity of the decoder. Usually a kind of suboptimal decoder (e.g-., multistage 
5 decoder) is used for multilevel codes. An example of a simple multistage decoder 20 is 
shown in FIG. 2. The decoder 20 employs a separate decoder (e.g., 22, 24, 26) for 
each of the codes Cu C2..... Cl. The decoding process starts with the decoding of Ci, 
based on the received signal. The decoding of Ci, for i = 2, 3, L is based on the 
received signal and the estimated codewords of Ci, C2, Q.u without using any 
1 0 information related to the codes Cj+i, . . . , Cl. 

The multistage decoder 20 is referred to as a suboptimal decoder because, 
during the decoding of a component code, there is no information regarding the 
component codes at the higher partition levels. For example, decoder 22 does not 
have information regarding the component codes output from decoder 24. The lack 
15 of this information can lead to an increase in the error coefficient of the code (and 
other elements in the spectrum of distances as well). Furthermore, errors might 
propagate through the decoders of the components codes, causing an increase in the 
average bit error rate (BER). However, the reduction in coding gain, due to 
suboptimal decoding, is small, whereas the reduction in complexity is substantial. 

20 

Parallel Concatenated Codes 

Reference is now made to FIG, 3, which is a block diagram of a simple parallel 
concatenated encoder 30. In this encoder, a block 32 of N bits is encoded twice, once 
by the encoder ei 33 (note that the notation E| is reserved for the multilevel encoder) in 
25 order to produce a codeword of Ci, and again, after interleaving, by the encoder 62 34, 
in order to produce a codeword of C2. The encoders 33 and 34 create two blocks 35 
and 36 of rl and r2 redundant bits, respectively. It should be appreciated that more 
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than two encoders 33, 34 can be used. However, additional encoders will generally 
require an additional interleaver 38, for each additional encoder. ' Thus, N encoders 
will generally require N-1 interleavers. Furthermore, there is generally no restriction to 
use dififerent encoders, and frequently c, and Cj. are the same binary codes. These 
5 codes may be block or convolutional codes. When a memoryless channel (e.g., 

AWGN channel) is used, the order of the transmitted bits is not important - e.g., each 
information bit can be followed by a redundant bit. 

The encoder 30 shown in FIG. 3 creates a + /•, + /-j, N) block code, where 
+ ri + ^2 is the length of this code. Since the information bits are transmitted only 
10 once, and are part of the codewords, e, and 62 must be systematic encoders. In case of 
block codes it is straightforward to construct a systematic encoder, many widely used 
block codes the generator matrix of the systematic code are well known. Note that a 
codeword of a parallel concatenated code consists of several codewords of ci and C2. 
Reference is now made to FIGS. 4A and 4E, which illustrate a non-systematic 
15 convolutional encoder 40 and a systematic convolutional encoder 42, respectively. 
Convolutional codes are conventionally implemented in a non-systematic form, as 
shown in FIG. 4A, where Gi {gu} and G2 {^2;} are the generators of the rate-half 
encoder. In the recursive systematic encoder of FIG. 4B, the input is divided by Gi 
and multiplied by G2. 

2° will be appreciated that both systematic and non-systematic encoders 

generally have the same trellis structure. However, the outputs produced by these 
encoders are different, even though the input provided to both encoders is the same. 
For instance, if the sequence (... 0 0 1 0 0...) is provided to the non-systematic 
encoder of FIG. 4A, then the outputs and ^uc'have a finite number of bits with the 

2 5 value " 1 " (this number is determined by the constrain length of the code). On the other 
hand, if the same sequence is provided to the systematic encoder, then a large number 
(generally limited by of bits with the value "1" will be provided at the output ^ik. 
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As is known the nonsystematic 40 and the systematic 42 encoders are a finite impulse 
response (FIR) and an infinite impulse response (IIR) systems, respectively. This 
property of the recursive systematic code makes it an important ingredient in the 
design of parallel concatenated codes, where the performance strongly depends on the 
5 spectrum of Hamming distances of codewords with small Hamming weight in the 
information block. It vAll be appreciated that the Hamming weight of a codeword 
refers to the number of "l"s in the codeword, where the Hamming distance between 
two codewords refers to the number of changed symbols (fi-om "0" to "1" or from "1" 
to "0") in two codewords. 

10 A parallel concatenated code can be treated as a block code, even when 

encoders 33 and 34 of FIG. 3 are convolutional encoders. For instance, if the encoder 
42 shown in FIG. 4B is substituted for the encoders 33, 34 of FIG. 3, then a (3N,N) 
block code with rate 1/3 is obtained . A zero state is assumed at the beginning and at . 
the end of the block. Since the constituent codes are recursive, it is not sufficient to 

15 terminate the block by setting the last A/ bits of a sequence lO zero. Kowever, 

relatively simple methods for trellis termination are well known, as for example in 
"Turbo Codes for PCS Applications", by D. Divsalar and F, PoUara, Proc of the ICC 
'95, Seattle, Washington, June 1995, and need not be discussed herein. 

From Shannon theory, it is observed that increasing the length of the codeword 

2 0 (or the constraint length in case of a convolutional code) leads to better performance. 
That is, by adding fijrther redundancy, a lower bit error rate (BER) is achieved. 
However, the complexity of maximum-likelihood (ML) decoding algorithm increases 
with the length of the codeword. Since codewords of parallel concatenated codes are 
relatively long, parallel concatenated codes have the potential of being good codes. 

25 The performance of a code is determined primarily by the spectrum of Hamming 

distances, which will be discussed in more detail below. Also, the interleaver (see FIG. 
3) plays an important role in determining the performance of a parallel concatenated 
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code. A ML decoding of a parallel concatenated code is impractical, because of the 
computational complexity involved. By taking the advantage of the structure of a 
parallel concatenated code, however, the decoding can be broken into simpler 
decoding steps, such that the total decoding complexity may be made lower than that 
5 of a ML algorithm. 

Referring now to FIG. 5, a suboptimal decoder 50 for the parallel concatenated 
code encoded according to FIG. 3 is presented. This decoder consists of decoders 51, 
52 for ci and Cj, and an interleaver 53 which is designed to correspond to the 
interleaver 38 used in the encoder 30. The inputs to decoder 51 are the received 

10 information bits, jk, and the received parity check bits, y^,, created by encoder 33. 
Decoder 51 is a soft in soft out decoder, aimed at providing soft (i.e., weighted) 
information to decoder 52. It is well known that soft decoding, performed by decoder 
52, yields performance that is better than that of hard decoding. For example, if Prfd^ 
= /lobservation], / = 0, 1, be the a posieriori probability of the data bit a^, the 

15 logarithm of the likelihood ratio (LLR) is the desired output of decoder 51, and is 
given by: 

J _ Pr[dk - l\obser vatjo?7] 

^''-^°^Pr[d. = 0\observationr Equation (1) 

The maximum a posteriori (MAP) algorithm is the optimal soft in soft out 
2 0 decoder. A modification of Bahl's algorithm is known for the recursive systematic 

codes. The computational complexity of the MAP algorithm is higher (especially when 
a long interleaver is used) than that of the Viterbi algorithm, which is unable to provide 
the desired soft output. Modification to the Viterbi algorithm, called soft output 
Viterbi algorithm (SOVA), has been suggested in literature, and is known by persons 
25 skilled in the art. This algorithm produces information regarding the reliability of the 
received data. From Equation (1) above, it is easy to see that when the difference 
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between Pr[d^ = 1 |observation] and Pr[t/k = Ojobservation] is small, the LLR of Lu, is 
close to zero. Such a small difference indicates small reliability of the information 
related to the k-th bit. If l|observation] > Pr[^ic=0|observiation], then Lik has a 

positive value, which increases with the reliability of the measurement. Namely, the 
5 desired soft output can be approximated by multiplying the hard decision with an 
estimation for the reliability of the measurement. In the SOVA the reliability is 
evaluated by measuring the difference between a survivor path and the path that has 
the closest accumulated metric. As this difference increases the reliability of the 
measurement increases, since the difference between the accumulated metrics of the 
10 survivor path and that of the remaining paths increases. 

As illustrated, decoder 5 1 in FIG. 5 is not using the data related to the parity 
check bits of C2. This information is instead utilized by decoder 52, which performs a 
soft decision. The foregoing decoding algorithm is not optimal,^ since decoder 51 is 
not using all the available information {e.g., not using >'2k). Accordingly, the 

15 Derformance can hp imnrnvpH hv 5;nh<;titntina (\e^^r.n(\p'r S7 fnr a cnft rMitnnt /^oz-r^HoT- 

i _ ^ . _^ ^ _ » WAV wwv|>«.*l. M>^W\^V*^X, 

similar to decoder 51. The soft output corresponding to the k-th information bit can 
be expressed in a sum of the input 7k and an extrinsic information, denoted by W^, (See 
FIG. 6). The extrinsic information is a function of the redundant information 
introduced by the encoder. Namely, represents the improvement (due to the parity 

2 0 check bits) obtained in the soft output. By using a feedback loop 62, the extrinsic 
information produced by decoder 52 is available to decoder 51 (see FIG. 6). The 
additional information provided by the feedback loop 62 improves the performance of 
decoder 51, and consequently the performance of decoder 52 {i.e,, the decoder's 
output). Using computational methods that are known, it can be deduced that the 

2 5 iterative decoder presented in FIG. 6 achieves performance that is close to the 

Shannon limit (and consequently are close to the performance of ML decoding) after a 
few iterations. 
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The Impact of the Interleaver on the Performance 

Interleaving algorithms and techniques were introduced to the art of 
communication in order to increase the noise immunity of systems that are subject to 

5 burst errors. Interleaving algorithms define the relations among data sequences of 
different codev^ord blocks on the transmission channel, and interleaving techniques 
define the implementation methods to achieve these relations. Examples of 
interleaving algorithms are block interleaving, and convolutional interleaving. 
Examples of interleaving techniques, are data interleaving and code interleaving. 

0 In data interleaving^ information is first applied to an encoder, and the 

encoder's output is interleaved. In code interleaving (a rarely used technique), the 
information is interleaved first and then applied to the encoding process. An 
Interlaced Interleaving technique is also known in a block interleaved arrangement, 
where the incoming information is gated to multiple encoders. The Interlaced 

5 Interleaving approach is an example of code interleaving wherein the information is 
interleaved first and then encoded. In effect, this method de-interleaves the 
information, routes the de-interleaved information to encoders, then re-interleaves the 
information and the redundant symbols. An understanding of interleaving algorithms 
can be gained by looking at the movement of data symbols in the time domain, or by 

) viewing the process of storing data in a matrix in accordance with one procedure and 
retrieving it in accordance with another procedure. 

In block interleaving, a block of data is rearranged to insure that consecutive 
symbols in the block prior to interleaving are not adjacent in the block after the 
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interleaving. For example, data is written into the block by row, but read from the 
block by column. A clear characteristic of block interleaving is that the interleaved 
data can be separated into blocks of consecutive symbols which correspond to blocks 
of consecutive symbols in the uninterleaved data. The difference between the two 
5 corresponding blocks is in the sequence of symbols in the blocks. 

For purposes of explanation, assume each block includes D groups 
(codewords) of N symbols each. Since block interleaving rearranges the sequence of 
symbols, it is understood that a delay must be introduced in order to allow later 
arriving symbols to appear earlier in the interleaved output. To a first degree of 

10 approximation, in block encoding, the interleaved data of each block can be obtained 
only after the entire block of data has arrived. The interleaving of data is accomplished 
by taking consecutive symbols of each codeword, and dispersing them at intervals of D 
symbols (the value of D being the interleaving depth). Thus, the first symbol of the 
first codeword in becomes the first symbol out, but the second symbol of the first 

15 codeword in becomes the D+1 symbol out, the third symbol of the first codeword in 
becomes the 2D-*-l symbol out, etc. 

As indicated above, the interleaving described above is block interleaving, 
where one can identify a block of input data and a corresponding block of output data. 
The output block contains only the data of the input block. 

2 0 In convolutional interleaving, no output block of contiguous data signals can be 

found that corresponds to an input block of data, in the sense of containing only the 
data of the input block. Instead, in convolutional interleaving, where the codeword 
length N is 11 symbols and the interleaving depth D is 5, it may be said that input block 
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supplies data to an interleaved output symbol sequence of length NxD, but it supplies 
only some of the symbols, and it also supplies data to a following symbol sequence. 

While interleaving methods and techniques are known in the art, a further 
understanding of interleaving may be obtained from U.S. Patent Application Ser. No. 
5 08/469,558, filed June 6, 1995, and assigned to the assignee of the present invention, 
which application is hereby incorporated by reference. 

To illustrate mathematically, let A^j be the number of codewords (in Ci) 

generated by an input information word of Hamming weight >v, where the parity check 
bits have the Hamming weight j\ The overall Hamming weight of such a codeword is 
10 w==viH7. The input redundancy weight enumerating function (IRWEF) is given by: 



A random interleaver (sometimes called uniform interleaver) maps a given input word 
of weight w into all distinct j permutations of it with an equal probability. When a 
random interleaver is used, the IRWEF of the parallel concatenated code (PCC) is 



Let the coefficient D„u corresponding to the codewords with an overall weight 
of m, be defined as follows: 




Equation (2) 



15 given by: 




Equation (3) 
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D„= Equation (4) 

j-^w — m 

By using the union bound, the bit error rate (BER) of ML decoding can be upper 
bounded by: 



^b-^ ^--I'^J' Equation (5) 



m 

5 where A is the Euclidean distance between the signal points in the signal constellation, 

1 r°° — 

a^is the variance of the AWGN, and Qi^) = J ^ ^ ^ * The union Bound is 

tight for moderate and large SNR, normally when the BER is smaller than 10^. For 
low SNR the tangential upper bound or the tangential sphere bound can be used. Note 
that the set of coefficients {D„,} plays an important role also in these bounds. 

10 Using known computational methods, it can be deduced that the BER reduces 

as the value of the elements in the set {D„,} decreases. Thus, it is observed that each 
coefficient Z>m decreases when the length of the. interleaver increases. Therefore, 
improved performance can be achieved by employing a random interleaver with a long 
delay. In comparison with the constituent codes, the parallel concatenated codes, 

15 having a random interleaver, provide only small improvement in the asymptotic coding 
gain. This observation is based on the fact that only a small increase in the nninimum 
distance of the constituent codes is obtained by this parallel concatenated encoding. 
Most of the known parallel concatenated codes use long random interleavers, designed 
to achieve large coding gain at BER in the vicinity of 10*^. Thus, it is seen that the 
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random interleaver is not the optimal interleaver for the case where the length of the 
interleaver is restricted and the desired BER is very small (e.g-., 10*^ or 10"*). 

In accordance with the concepts and teachings of the present invention, a 
structured interleaver is substituted in place of the random interleaver. Particularly 
5 when a short interleaver is used, it has been realized by the present invention that the 
desired BER is small, as compared to the use of a short, random interleaver. It is 
observed that error correction codes have two important parameters that determine the 
performance of the code. These parameters are the minimum Hamming distance (the 
minimum value of w), and the set of coeflEicients {D^} (especially the coefficient 
10 corresponding to the minimum distance, called error coefficient). For moderate as well 
as high SNRs the minimum distance is the dominant parameter, whereas the effect of 
the error coefficients is treated as a less important factor in the conventional coding 
theory. For instance, a known rule of thumb is that the coding gain for BER in the 
vicinity of 10"^ decreases by 0,2 dB when the error coefficient is multiplied by a factor 
15 of two. In fact, that this rule holds for many coding structures. However, in the 
known parallel concatenated codes excellent performance is achieved due to the 
tremendous reduction in the error coefficients, obtained by utilizing a very long 
random interleaver. 

As an illustrative example, consider the use of a parallel concatenated code that 
2 0 consists of two identical convolutional codes with M = 2. If the generators Gi=5 and 
^2=7 (in octal base) are used, then codewords (of the constituent code) with the 
minimum distance have the input sequence (.,.00 1 0 1 00...). For these codewords 
w=2, /77=5, and = N-2. It is readily deduced that the minimum distance of the 
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parallel concatenated code is 8 (the input weight is two and the redundant bits of each 
constituent code contributes a weight of three). It is further deduced that 
4(yV-2)- 

A = (jv- * ^'^^^^ N»2, £)8=4/N {e,g,, for - 10,000, £>s=0.0004), which is a 

very small error coefficient. If the generators Gi=7 and G2=5 are used, then a 
5 codeword (of the constituent code) with the minimum distance has the input sequence 
(...0 0 1 1 1 0 0..,). For this case w=3, /72=5, and A^^^ ^ N -2. It can thus be deduced 

that the minimum distance of the parallel concatenated code is 7 (the input weight is 

three and the redundant bits of each constituent code contributes a weight of two), and 

\%{N-2) 18 
that A = _ ^^^2 • Since N»2, e,g,, fox N=\0, 000, D, ^ \.%x\0'\ 

10 which is much smaller than the error coefficient obtained for the case where Gi-5 and 
G2=7. It is therefore appreciated that the error coefficient decreases as the weight w 
(of the codewords with the minimum distance) increases. 

As explained above, the advantage of the random interleaver is in reducing the 
a priori probability of the transmitted sequences, and consequently reducing the error 

15 coefficients {25^}. Recall that when Gi=5 and G2=7, the input sequence (;.., 0 0 10 1 
0 0,...) yields a sequence of redundant bits with Hamming weight of three by the 
encoder e\ (See FIG. 3). Using a structured interleaver, however, avoids applying this 
same input sequence simultaneously to both encoders 33 and 34, and a reduction in the 
minimum Hamming weight of the parallel concatenated code is achieved. 

2 0 Referring now to FIG. 7, a diagram of a simple block interleaver 70 is shown, 

where the data is written in Ni rows of a matrix with N2 colunms (N = NxNi), and read 
column by column. In FIG. 7 it is also shown that when the "worst" input sequence is 
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provided to encoder 33, the input sequence to encoder 34 contains 3N2-2 zeroes 
between two ones. Namely, the recursive encoder 34, will create in this case a 
sequence with a relatively large Hamming weight, and consequently a parallel 
concatenated codeword with large weight obtained for an input sequence with a 
5 weight of two. 

It is observed that a block interleaver may suffer from low minimum Hamming 
distance, due to input sequences with ones at the end of the block. For instance, if a 
single "r* is located at the end of the input sequence, then a short no sequence is 
obtained at the end of the redundant sequences created by encoders 33 and 34. One 

10 solution to this is simply to utilize a reverse block interleaver 80, as shown in FIG. 8. 
In this reverse block interleaver 80, the data is read from the end of the last column 
through a path which is opposite to the path used for the regular block interleaver. If 
a single one is located at the end of the input sequence (FIG. 8), then a short non-zero 
sequence will be created by encoder 33, whereas the input of encoder 34 is a single "T 

15 followed by "0"s, and consequently a long non-zero sequence will be created by 
encoder 34. Therefore, when a reverse block interleaver is employed, a large 
Hamming distance is obtained for the foregoing input sequence. 

FIG. 9 is a diagram illustrating the "worst" reverse block interleaver 90, when 
Gi=5 and G2=7. This information matrix includes two columns with "T's, creating 

20 two self-terminated sequences at the output of encoder 34. In this situation, the 

overall Hamming weight of the parallel concatenated codeword is 16, and is calculated 
as follows: The Hamming weight of the information sequence is 4. The Hamming 
weight of the redundant bits created by the input (...0 0 1 0 1 0 0...) is 3. Since two 
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sequences of this kind are provided to the encoders 33 and 34, the total Hamming 
weight of the redundancy is 3x4=12. 

When the reverse block interleaver is utilized, the minimum Hamming distance 
can be increased by using the generators Gi=7 and G2=5. In this case, the constituent 
5 encoder produces a redundant sequence of weight two in response tb the input (...00 
1 1 10 0..,), whereas the input (...00 1 00 1 0 0.,.) yields a redundant sequence of 
Hamming weight four. Although the latter sequence is not corresponding with the 
minimum Hamming weight of the constituent code, the parallel concatenated 
codeword with the minimum weight includes this input sequence. The "worst" input 
10 sequence is shown in FIG. 10. In this sequence, the minimum Hamming weight of the 
parallel concatenated code is 4x4+4=20 (where the first term represents the 
contribution of the redundant sequence). It is easy to see that the corresponding error 
coefficient is given by 

4 

D^^ = —{N,~ 3){N^ - 3) < 4. Equation (6) 

15 When Gi=7 and G2=5, a codeword with a weight of 21 is obtained for the 

information sequence shown in FIG. 1 1. Similarly, when Gi=7 and G2=5, a codeword 
with a weight of 22 is obtained for the information sequence shown in FIG. 12. The 
coefficient D21 is given by: 

9 

2 0 D^,= — (//j - 2)(N^ - 2) < 9, Equation (7) 

Since a codeword with a Hamming weight of 22 is also obtained by transposing the 
information matrix of FIG. 12, D22 is given by: 

A2 = ^[(^t - 2)(A^2 - 3) + (A^^ - 3)(N, - 2)] < 12. Equation (8) 



21 



wo 98/11671 



PCT/US97/16106 



It will be appreciated that the block interleaver does not have to be long in 
order to obtain the error coefficients calculate above. In fact, it can be shown that N 
may be less than 200. For the case where a random interleaver is used, the set (Dm) 
can be calculated according to Equations (3) and (4). For Gi=5 and G2=7, with 
5 //= 1 ,000, and 1 0,000, the resulting error coefficients have been computed and 
listed in prior art publications, such as "Unveiling Turbo Codes: Some Results on 
Parallel Concatenated Coding Schemes"," S. Benedetto and G. Montroai, IEEE Trans. 
Information Theory^ vol. IT-42, pp. 409-428, March 1996. Using the union bound of 
Equation (5), the BER can be evaluated for low as well as for moderate BER. FIG. 13 

10 is a graph 130 shoving the resulting (computed) BER versus SNR for the reverse 
block interleaver, and random interleaver with lengths of 1,000 132 and 10,000 134. 
As shown, the random interleaver of length 10,000 134 achieves a lower BER, for a 
given SNR, than the random interleaver of length 1,000 (as would be expected). At 
extremely low SNB. values, both random interleavers achieve lower BER than that 

15 achieved by the reverse block interleaver. However, at SNR values above 

approximately 1 .5, the structured reverse block interleaver achieves a lower BER than 
both random interleavers 132 and 134. It is appreciated that the length of the reverse 
block interleaver is shorter then the two random interleavers. 

Thus, from F;IG. 13 it is deduced that, for low SNRs, there is an advantage to 

2 0 the random interleaver over the reverse block interleaver. However, the parallel 

concatenated code with the structured interleaver performs better as the desired BER 
decreases. For BER smaller than 10"^, the coding gain achieved by the reverse block 
interleaver is larger than that obtain by the random interleaver with N = 1,000. 
Advandageously, the reverse block interleaver performs batter than a random 

2 5 interleaver with a length of 10,000 when the BER is smaller than 10*''. The advantage 
of the reverse block interleaver over the random interleaver increases as the desired 
BER decreases. As previously discussed, the minimum Hamming distance of the 
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parallel concatenated code with a random interleaver, Gi=5, and G2=7 is 8, whereas 
the minimum distance of the parallel concatenated code with reverse block interleaver 
Gi=7, and G2=5 is 20. Therefore, the asymptotic coding gain, obtained for the reverse 
block interleaver, is larger than that of the random interleaver by 10 Iogio(20/8) = 3.98 
5 dB. Clearly, the asymptotic coding gain does not depend on the length of the random 
interleaver. This phenomenon explains the results in FIG. 13, where the advantage of 
the longer random interleaver over the shorter random interleaver reduces as the 
desired BER decreases. 

FIG. 14 shows simulation results of parallel concatenated codes with block 144 

10 and reverse block 142 interleavers. Both simulations are based on two 4-state 

constituent codes decoded by a suboptimal iterative decoder (using the approximation 
log(e^* + e^)=max(Li, L2), with 8 iterations. The performance of the ML decoder for 
the case of reverse block interleaver is also illustrated as 146. Thus, from FIG. 14 it is . 
observed that the performance of the iterative decoder is close to that of the ML 

15 decoder, especially for small BERs. Thus, the advantage of the reverse block 

interleaver over the block interleaver is clear from the simulation shown in FIG. 14, 

As previously mentioned, a significant advantage achieved through the use of a 
structured interleaver is the ability to submit less than all information bits to each of the 
encoders. While it is generally desirable to encode all information bits, such encoding 

2 0 need not be performed at a single encoder. It will be appreciated that the same does 
not hold true when a random interleaver is used- Indeed, whenever a random 
interleaver is used, if fewer that all the information bits are submitted to each of the 
encoders, there is a statistical probability that one or more of the information bits will 
not be encoded. In contrast, by using a structured in accordance with the present 

2 5 invention, the interleaver can be designed to ensure that all information bits get 

encoded, even though only a portion of the information bits may be submitted to each 
encoder. 
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Multilevel Codes with parallel concatenated Codes as Component Codes 

Turning now to the evaluation of multilevel codes (See FIG. 1 illustrating a 
simple multilevel encoder), having parallel concatenated codes as component codes, let 
5 Ai be the minimum Euclidean distance between elements in the set .Si, where is the 
minimum Hamming distance of the binary code Q. It is seen that the minimum 
squared Euclidean distance between codewords in a multilevel structure is given by: 

= min(A\d^'\^,d^^\.,.,A\_,d^''^) . Equation (9) 

10 Furthermore, any product A._, ^d^'^ (where dj'^ is a possible Hamming distance in the 

component code of the multilevel code Cj) is a possible Euclidean distance between 
codewords in the multilevel code. Since Si,\ is a subset of Si,Ai.i<Ai. Therefore, In 
order to obtain multilevel code with large coding gain, the correction capability of Cm, 
should be larger than that of Q. Conventionally, L=2 is used, where in many well 

15 known multilevel structure C2 is simply a parity check code, such as the Leech half 
lattice and lattices from the Barnes- Wall family. 

Methods are known for evaluating the BER contributed by each component 
code. A design rule for multilevel coding is to approach a balance among the BER 
contributed by the component codes. Notice that this design rule ignores the affect of 

2 0 the errors that might propagate from the bits associated to lower partition levels to the 
higher levels (due to multistage decoding). However, for moderate as well as low 
BER, this affect is small. In order to compensate for the error propagation, the BER 
contributed by the bits related to the lower partition levels should be slightly smaller 
than that contributed by the higher partition levels. 
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For simplicity consider an equally spaced q-ary pulse amplitude modulation (q- 
PAM). Note that the results are applicable, to <f quadrature amplitude modulation 

and can be extended to other signal constellations. Let = A,._j ^d^^ , 
where the set {^*^} is the set of dominant Euclidean distances, associated with the i-th 
5 partition level. Let /: = 7, 2, ^x2'' be a symbol in the subset S\ with the a 

priori probability P^'^ (given that the symbol belongs to the subset S{). Let aj^'^(be the 
number of neighbors of that symbol in the subset S\.i. Notice that for the case of 
equally spaced q-PAM, a^^^ may have one of the values 1 or 2. It is then appreciated 
that the BER contributed by the i-th partition level is upper bounded by the following 
10 union bound: 



2a ^ 



wnere 



Equation (10) 



Equation (11) 



15 



and {D^'^} is the set of coefficients of G, defined in Equation (4). In case where all the 



signal points have the same a priori probability. 



»=1 ^ 



4" 



Equation (12) 



20 



From Equation (12) it can be deduced that for 8-PAM and 64-QAM 



Bi'' = D^*>1.75"^ , and 5f> =i5f>iy* , 



Equation (13) 
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It is seen that the bits associated with the first partition level require the larger 
correction capability. Therefore, we consider here multilevel codes with parallel 
concatenated code as C\. As an example assume a signal constellation of 64-QAM. 
FIG. 1 5 shows the results of a simulation (computed) illustrating the upper bound on 
5 Pbi, where Q is a PCCC vAthM = 2. The reverse block interleaver 152 and random 
interleaver discussed previously, with//=l,000 132, and A^=10,000 134are compared 
in FIG. 15. The set B^^^is calculated according to Equation (13), where the set of the 
error coefficients {Dl^^} of the random interleaver is given in "Unveiling Turbo Codes 
Some Results on Parallel Concatenated Coding Schemes"," S. Benedetto and G. 

10 Montroai, IEEE Trans. Informatioji Theory, vol. IT-42, pp. 409-428, March 1996, 

and is calculated in Equations (6)-(8) for the reverse block interleaver. From FIG. 15 
it is concluded that for BER smaller than 10'^, the reverse block interleaver is better 
than the random interleaver, even when the length of the random interleaver is 10,000. 
Recall that a similar result is dem.onstrated in the case of binary modulation with these 

15 parallel concatenated code (see FIG. 13). 

Clearly, the code C2 should preferably have correction capability which is 
smaller than that of Ci. However, a simple parity check code is generally not 
sufficient, because of the large correction capability obtained when Ci is a parallel 
concatenated code. On the other hand, in a multilevel structure with L=2, the uncoded 

2 0 bits will contribute an Euclidean distance of A2=15Ao. Namely, a minimum Hamming 
distance of 4 will be sufficient for C2. In order to reduce the decoding complexity, a 
parallel concatenated block code (PCBC), which employs a parallel concatenation of 
three parity check codes and two structured interleavers, is used. These interleavers 
are designed to ensure that two codewords of different constituent codes have only 

2 5 one information bit in common. It can be shown that the dominant Hamming distances 
d^^^ are (4, 6, 8), and the corresponding error coefficients d^^^ are (1, 72, 3096). The 

BER of Pb2 is calculated according to Equation (10) and Equation (13), and shown as 
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line 162 in FIG. 16. The BER contributed by the first partition level (where C\ is a 



PCCC with M = 2 and reverse block interleaver), and the BER for the uncoded 



transmission are also presented in FIG. 16 as lines 164 and 166, respectively. From 
FIG. 16 it is seen that for BER smaller than 10'^ a coding gain of about 5 dB may be 
5 achieved for this relatively simple structure. It will be appreciated that the real coding 
gain will depend in part on the performance of the suboptimal iterative decoder. 

The performance of a multilevel structure can be improved by increasing the 
correction capability of Ci. From the foregoing discussion, it will be appreciated 
that the convolutional code with Gi^l3 and G2=17 has the best spectrum of Hamming 
10 distances. FIG. 17 shows the resulting (as line 172) when Ci, is the foregoing 
PCCC. For comparison Pb2 and the BER for the uncoded system are also shown in 
FIG. 17 (as lines 174 and 176, respectively). In this case a coding gain of more than 
5.5 dB may achieved for BERs smaller than 10'^. 



15 description. It is not intended to be exhaustive or to limit the invention to the precise 
forms disclosed. Obvious modifications or variations are possible in light of the above 
teachings. The embodiment or embodiments discussed were chosen and described to 
provide the best illustration of the principles of the invention and its practical 
application to thereby enable one of ordinary skill in the art to utilize the invention in 

2 0 various embodiments and with various modifications as are suited to the particular use 
contemplated. All such modifications and variations are within the scope of the 
invention as determined by the appended claims when interpreted in accordance with 
the breadth to which they are fairly and legally entitled. 



The foregoing description has been presented for purposes of illustration and 
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What is Claimed is: 

1 . A method for coding an information signal comprising the steps of: 

encoding a portion of the information signal with a first encoder to 
generate a first set of r i redundant bits, where ri is an integer value of one or more; 

passing a portion of the information signal through a structured 
interleaver to generate an interleaved signal; 

encoding a portion of the interleaved signal with a second encoder to 
generate a second set of r2 redundant bits, where r2 is an integer value of one or more; 
and 

concatenating the information signal, the first set of redundant bits, and 
the second set of redundant bits to form an encoded output signal. 

2. The method as defined in claim 1, wherein the step of encoding a 
portion of the information signal includes encoding the entire information signal. 

3. The method as defined in claim 1, wherein the step of passing a portion 
of the information signal through a structured interleaver includes passing the entire 
information signal through the structured interleaver. 

4. The method as defined in claim 1, wherein the structured interleaver is a 
reverse block interleaver. 
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5. The method as defined in claim 1, wherein the structured interleaver is 
reverse block interleaver defined write to and read from a matrix of information bits 
according to the following form: 



Write 



XXX 
XXX 
XXX 



tRea^/, 



where the matrix may be of variable dimension and the 'x's denote don't care values. 



6. The method as defined in claim 1, wherein ri equals T2. 



7. The method as defined in claim 1, further including the steps of 

passing the interleaved signal through a second structured interleaver to 

generate an second interleaved signal; and 

encoding the second interleaved signal with a third encoder to generate 

a third set of r 3 redundant bits, where rs is an integer value of one or more. 



8. The rnethod as defined in claim 7, wherein the step of concatenating 
further includes concatenating the information signal, the first set of redundant bits, the 
second set of redundant bits, and the third set of redundant bits to form an encoded 
output signal. 
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9. An encoder for providing a multilevel encoded signal comprising: 
a serial to parallel converter having an input for receiving an input 

signal and a plurality of outputs; 

a plurality of encoders, wherein each of the plurality of encoders has an 
input and an output, the input of each of the plurality of encoders being associated with 
one of the outputs of the serial to parallel converter; 

a symbol selector having a plurality of inputs and an output, each of the 
plurality of inputs being associated with one of the outputs of the plurality of encoders; 
and 

at least one structured interleaver interposed between an output of the 
serial to parallel converter and the input of the associated encoder. 

10. The encoder as defined in claim 9, further including an output of the 
serial to parallel converter being directly associated with an input of the symbol 
selector. 
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